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AIDE AU REPERAGE TRIDIMENSIONNEL POUR LA CHIRURGIE DE 

LA BASE DU CRANE, 

L. ADAMS, A. KNEPPER, W. KRYBUS, D. MEYER-EBRECHT, G. PFEIFER, 

R. RUGER, M. WTTTE. 

RESUME : 

Dans le cadre d'un projei dc recherche interdisciplinairc, un systcme d'aidc au repcrage 
cridimcnsionncl pour la chirurgic dc la base du crane a 6i6 d^velopp^. En combinant des 
techniques dc capteurs dc position 3D, dc traitement d*iinages et dc visualisation 3D, 
rorientation et la position d'un outil chirurgical sont visualises sur un fcian. L*image dc 
la zone operec est ainsi synth6ris& k partir de coupes tomodcnsitom^triques (TDM). Ce 
systeme a ii£ utilise dans plus de 300 interventions en udlisant un capteur de position 3D 
electro-mecaniquc. deplace manuellement. Pour faciliter la manipulation, des techniques 
de mesurcs 3D sans contact ont etc etudiees. Nous avons d^veloppc un capteur 3D 
oprique, qui utilise un ensemble de sources lumineuses infra-rouges dispos^cs sur 
rinstrument chirurgical. Ces diodes lumineuses sont dctcctdes par des cameras CCD. 
Une precision de 1.5 mm a 6te obtcnue. Des diodes suppl6mentaires situees sur des 
points de reference sur la tete du patient pcrmettcnt dc calibrer le systcme sans aucun 
procede manuel et de suivrc automatiquement chaque diplaccmcnt du aine en effecmant 
la mise en correspondance avec le volume TDM. tJne configuration hardware specifique 
a etc developpec, elle permet de rcaliscr Taffichage dc la position dc Toutil sur des 
coupes TDM quclconques en temps reel. 

Mots des : Chirurgic assist^c par ordinateur, Chirurgic dc la base du crane, Mise en 
correspondance et Visualisation temps reel, Capteur oprique 3D, Guidage passif. 



ORIENTATION AID FOR HEAD AND NECK SURGEONS 
ABSTRACT : 

In an interdisciplinary research project an orientation aid for skull base surgery has been 
developed By combining 3D position measurement techniques, digital image processing 
and 3D display techniques the spatial position and orientation of an operating instrument 
is being visualized on a display, which shows a CT-generated view of the operation area. 
In more than 300 operations the system was applied by using a handguidcd 
electromechanical 3D coordinate digitizer. To yield improved flexibility and easier 
handling, non-contact measurement techniques were investigated. For this purpose we 
developeid an optical measurement system, which uses a set of infrared light sources 
mounted on the handle of a surgical instrument. These sources arc detected by array- 
cameras. An accuracy of 1.5 nun has been achieved. Further LEDs attached to the 
reference points on the head avoid the need of calibrating the measurement set-up by 
touching the reference points with the tip after each displacement of the skull. A special 
hardware was developed which permits the on-line combination of arbitrary cutting 
planes through the CT-volume with a pscudo 3-D representation of the head reconstructed 
from the CT-slices. 

Key Words : Computer Assisted Surgery, Skull Base Surgery, Real-time Registration 
and Visualisation, C^tical 3D Sensor, Passive System. 
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Orientation Aid for Head and Neck Surgeons 
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Lehrstuhl fiir McBtechnik, RWTH Aachen, D-5 100 Aachen 
Introduction 

Computer Assisted Surgery (CAS) shall introduce "instrument navigadon" into 
the surgeon's daily practice by providing three-dimensional position measurement 
techniques for the operation theatre. The surgeon's map will be a pseudo-three- 
dimensional presentation of the patient taken down from CT images. A measuring system 
capable of determining an instrument's position with respect to the patient will be the 
navigation guide. This system will also serve as a "flight-simulator" for realistic 
preoperative planning. 

Two major advances can be achieved by CAS: Before starting the operation, the 
surgeon can "walk" through the 3D image of the proposed operation area thus gaining a 
realistic impression of the region and its neighbourhood. During the operationi the 
surgeon can yield position information by simply pointing at an unidentified structure 
with the tip of a 3D coordinate digitizer. The respective CT sectional views are then 
displayed on the CRT with an indication of the position of the coordinate digitizer. By 
means of the appUcation of this orientation aid, safety and quaHty of the operation will be 
increased, time consuming procedures can be avoided and novel procedures become 
possible in surgery. 

The CAS-system has been appUed in more than 300 operations by using a 
handguided electromechanical 3D coordinate digitizer. Recently we developed an optical 
measurement system to improve the flexibility of the digitizer. The visuaHsation of the 
CT-images is now supponed by a special hardware which combines cutting planes 
through the CT volume with pseudo 3D images of the patient's head. 
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State of the Art 

In the course of operations at the skull base or in the region of the ears and the 
paranasal sinuses, intra-operative orientation can be extremely difficult due to individual 
variations of the padents' anatomy. The surgeon visually controls the position of the 
surgical instrument. Before and during the operation, several diagnostic imaging 
techniques are at his disposal for his orientation: X-ray films or CT image sequeftces 
which have been prepared by preceding investigations, are displayed on lighiboxes' 
However, because of lacking correlation between those images and his visual 
impression, even the experienced surgeon is sometimes not sure of the exact position of 
his mstmment. In exceptional cases, intra-operative X-ray screening is appUed. Radiation 
load -and two-dimensionaKty. however, restrict appUcation and benefu of this method. A 
continuous and reUable information concerning the position of the surgical instrument 
with respect to the anatomical structures is not possible this way. The same appUes for 
neurosurgical operations of deep seated introcercbral lesions, for which no landmarks are 
available. 

Stereotactic operations are done with the help of a localisation frame and an 
instrument holder. In die course of the operation, the position of a pathological brain 
process related to the fixed frame is presented on CT images which were prepared by 
means of preceding CT examinations. Then, an instrument holder is firmly connected to 
the frame. By moving die instrument along the straight padi of die fixed suppon, without 
further X-ray examinations, a sample of tissue is taken at the desired position, a local 
lesion is set or a radiating substance is implanted. It is the neurosurgeon's task to perform 
the linear displacement of the instrument along the patfi calculated by a computer. 
Disadvantages of die stereotactic method are die necessity to put the patient widi a frame 
into the CT-scanncr and die restricted handling of the instruments in die fixed frame 
during die operation. In order to improve die stereotactic procedure, to position and to 
advance an instrument, devices have been developed which automatize die linear 
displacement of die instrument by die application of robots [1][2][3]. Odier groups are 
working widi 3D coordinate digitizers [4][5][6][7][8]. 

All diese mediods suffer from one or more of die following shortcomings: 
• they only work punctually, 

the point of interest is approached along a linear padi. 

diere is a rigid linkage between die locaKsation frame fixed to die skull and die 

displacement of the surgical instrument. 
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the displacement of the instniment is neirfier feedcd back nor visuaUy presented in 

the context of the operadve field. 

the practical use is limited because of the low accuracy. 



CAS-Principle 

The deyelopment of the CAS system at the Aachen University of Technology 
predominantly aimed at supponing ENT surgery [9]. For this purpose, the CAS methods 
had to satisfy the following requirements: 

the objert to be operated and its environment shall be condnually presented on a 

display system, 

the position of the surgical instrument shall simultaneously be faded into the 
display presentadon of the operadve field, 

a frame attached to the skuU shaU be avoided and the surgeon shaU be permitted to 
change the position of the skull during the operation. 

The above requirements were fulfilled by combining 3D presentation methods 
with 3D coordinate measurement techniques. The surgeon uses a stylus the position of^. 
which can be measured. He tips onto points within the operative field the position of 
which he needs to know. Before the operation, a 3D data model of tiie object have to be 
prepared based on a complete set of computer tomograms. The measured coordinates are 
then correlated with the computer model to display the CT-sUces of the object with faded- 
in position markers. Fig. 1 schematically shows the individual steps necessary for the 
application of the CAS-system. 



CT*sc3nnin| of 
the patiou's head 



3D view of die 
operaxion vet and die B^::^:<^;$1 
instruments posiuoo 





CT-slic 



voxel model 



Fig. 1: CAS principle 



At first radiopaque markings are attached to the patient's skull and a CT 
investigation is performed. After transfering the images into the CAS-system, the 
coordinates of these reference points are determined in the coordinate system of the CT 
scanner. During the operation the position of the patient is rcgistrated by measuring the 
position of the reference points at his head. The derived two point-sets permit the 
calculation of the ^transformation between the coordinates of the measurement device and 
the patient's voxel model. After this the position of the surgical instrument can be faded 
into a presentation of the operative field, 

3D Coordinate Measurement 

The measurement system we have been using so far is an electromechaincal 3D 
coordinate digitizer, which has been developed especially for the CAS application. The 
principle of construction looks similar to an industrial multi-joint robot arm (fig. 2). 




Fig. 2: electromechanical 3D coordinate digiti[zcr 

^ The individual aim segments of the device are connected among each othcr'W 
of rotary joints. The angles of each of the rotary joints are measured by mean/of 
.iiDtary encoders so that the coordinates of the tip of a stylus mounted to the arm can be 



- 414 - 



calculated. Our measuring system has six joints and its first two arm elements are 
counterbalanced. We have applied optical increment encoders for shaft angle 
measurement The accuracy of the system is better than ±0.5 mm in a measuring volume 
of 0.3 m3. It has now proved its feasibility in more than 300 operations (fig. 3). 



In the meantime we investigated also measurement methods which do not require 
any mechanical linkage to the instrument Thfesc methods offer two major advantages: 
The surgeon can handle his instrument in the usual way. 

The position of special markcn, which have been attached to the reference points 
on the patient's skull, can be measured continuously. Because there is no further need for 
the surgeon to measure the patient's position by pointing to the reference points, the risk 
of unnoticed patient's movement is eliminated 

An assessment of a variety of different methods for a non-contact coordinate 
measurement (optical, ultrasound, magnetic fields) led us to tiie conclusion, that the 
triangulation of infrared light sources by means of fixed CCD cameras seemed to be the 
most robust approach. Such a system was developed. Its operation will be described 
below. 





F\^. 3: CAS onerarion 
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Optical 3D Measurement 

The developed system deiermines the position and orientation of the patient's 
head and the handle of an instrument by infrared diodes, which are attached to both. The 
diodes are detected by cameras and the position of the instrument handle and the patient's 
head can be calculated by triangulation (fig. 4). 




Fig. 4: principle of stereoscopic vision 

As the geometrical arrangement of the diodes is known, the position of the body 
can be determined even if none of the attached diodes is detected simultaneusly by both of 
the cameras. This is a necessary precondition for the practical application of the system, 
since the radiation angle of the emitters is limited and some of the emitters may be 
screened off. 

Fig. 5 shows the developed prototype system. We use three cameras to ensure 
that the acquisition of data remains still possible even if one camera is screened off (e.g. 
by the surgeon). The cameras are equipped with standard CCD array sensors. These 
sensors have no geometrical distortions but their low resolution (570 x 600 pixels) 
requires a calculation of the image coordinates with subpixel accuracy. Our investigations 
have shown that the resolution of the detectors can be increased to ±0.06 pixel, leading to 
an accuracy of less than ±0.4mm for the position of one emitter (measurement space of 
600 X 600 X 400 mm^, arrangement of cameras as shown in fig. 5). 

Since the CAS application requires realtime measurement, the pixel data of the 
cameras (300 kByte per frame) are analyzed online by dedicated preprocessors. A 
reduced data set is given to a standard microprocessor which computes the position of the 
object To perform these calculations, all the geometrical parameters of the cameras (e.g. 
focus length, distances between cameras, orientation of cameras) have to be known. 



Fig. 5: prototype system 



In order to deternune these parameters we use a cage with 57 infrared emitters 
mounted on. Since the positions of these emitters are exactly known, all required 
parameters can be derived from their image coordinates. No caHbration of the system is 
required if the whole camera arrangement is moved, because only the relative posinons 
between the instrument and the patient have to be determined. This offers the possibU.ry 
to move the measuring space if required (e.g. by fixing the cameras to the opera 
lamps). 

The instrument handle is equipped with five infrared diodes as shown in fig- 6. U 
is seized by the distance between the two glass tubes. The shown arrangement h^^^^^ 
chosen to obtain large radiation angles of the emitters as well as long distances 
them. 

A large distance between the detected emitters is mandatory for the ^^"^"'^^"^ 
the exact position of the instrument's tip. The instrument handle is impermea c 
be steiiHzed by sterilizing solution, gas steriHzer or iiradiarion. 



The system was tested successfully in the operation theatre under simulated 
operation conditions. 



in&ared 
diodes 




infirared 
diodes 




Fig. 6: instrument handle 

3D Visualisation 

To present three-dimensional objects, there are two basically different 
approaches. In case of the first approach, the boundaries of anatomical structures 
(organs, bones, blood vessels etc) are determined on the given two-dimensional CT 
images. Then, the so gained structures are translated into a 3D model of the object's 
surface which can be presented as if viewed from any desired point of view. The three- 
dimensional impression is conveyed to the observer on account of an appropriate 
shading. Specific optical characteristics (colour, transparency) which amclorate the 
distinctness of the presentation can be related with individual objects. The disadvantage 
of these methods is the necessity of defining the boundaries of the objects, i.e. the 
segmentation of the CT images. The process of segmentation leads to a reduction of the 
informative content of the CT images to borderlines of organs and tissues. 



In case of the second group of methods, the complete information content of the 
primary data is maintained. As a basis for the prcsentadon, the entire set of 3D data is 
employed rather than only the surfaces of objects. Each pixel of a CT image is 
understood as a one volume element (voxel) of a cube of voxels which represent the 
object space. Visual representations of the volume interior can be presented by 
appropriate sectional views across the volume. Within these sectional views, the original 
CT sectional views may also appear. 

After several experiments and thorough discussion concerning the best 
understanding of the display presentation, we decided to present the operative field by 
three perpendicular sectional views through the operative volume. This type of presenting 
a 3D object on a 2D medium is well known from mechanical design. Incidentally the 
surgeon will also get the accustomed presentation of CT images this way. In addition to 
the usual axial CT presentation he also gets the presentation of the corresponding 
perpendicular layers (sagittal or frontal display) now. These images arc generated by the 
calculation of three perpendicular sectional views through the voxel niodel (fig. 2). An 
interpolation between given CT-scctions is not necessary because the size of the voxels (1- 
X 1 X 2 mm) is small enough for a smooth image representation. The intersection of the 
three sectional views corresponds with the position of the coordinate digitizer within the 
operative field and is marked by a cross-hairs cursor on each sectional view. 

The following equation describes die connection between the image of the display 
and the sectional views within the voxel model: 



By means of this transformation equation, sectional views through the voxel 
model can be calculated under any desired angle. This is necessary if structures of objects 
are not distinctly presented on account of the given three peipcndicular sectional views. 
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The three-dimensional understanding is conveyed to the oberserver on account of 
the fact that the secdonal views can be rapidly moved forwards and backwards along any 
desired axis. 

Recendy we developed a method to show not only the position of the tip but also 
the shape of the instrument within the operative field (fig. 7). This became necessary 
because of the new application of the CAS system in the field of neurosurgery. For an 
ENT surgeon it is most imponant to know where the- up of his instrument is. A 
neurosurgeon is primarily interested in finding the best way to his operative field. This 
becomes possible with the information about the direction of the instrument. On the 
display the cross-hair cursor is replaced by a graphical representation of the instrument. 




Fig.7: CAS display 

In addition to the three perpendicular views of the operative field, a double 
oblique cutting plane through the voxel model is displayed on the screen. Three 
alternatives to define this cutting plane are offered. 

In the first case the cutting plane is parallel to the instrument plane. Strucmres 
behind and around the instument are well visible. In the second case the cutting plane is 
perpendicular to the jxis of the instrument The distance between pointer tip and cutting 
plane is variable, so that the path to the operative field can be easily planned. The third 



possibility is like a homing flight. At first a destination point, for example a brain rumor 
is defined. Then the cutting plane is defined as a plane perpendicular to the instrument, 
which the destination point is placed on. The destination point, the point of intersection of 
the axis of the instrument and the cutting plane are marked with a cross-hairs cursor The 
surgeon has found the right direction to the tumor when the two cursors meet each other 



When fading the projection of the stylus into a double oblique cutting plane 
through the volume model, as decribed above, the identification of the position and 
orientation of this plane appeared to be difficult. This problem has been solved bv 
combining these cutting planes with a precalculated pseudo 3-D image of the patient's 
head (fig. 8). 




Fig. 8: double oblique cutting plane with additional surface modelling 

To combine a sectional view of an abitrary cutting plane with a gray-level shaded 
surface image without loss of speed, transformations to display the cutting plane and the 
surface image are calculated simultaneously. This makes it possible to decide for cvco 
pbccl of the destination image, whether surface image or cutting plane is to be displayed 

This decision is made by comparing the virtual depth of the surface and the 
cutting plane. On the sections of the image where the surface is located behind the cutnne 
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the surface pixel has to be shown. Is, however, the cutting plane located behind 
plane, ^ _ 

the surface, the corresponding CT-voxel has to be displayed. 

The virtual depth of the cutdng plane is a linear fundon of the two display 
coordinates and can be computed on the fly. Since the depth of a natural surface can not 
easily be expressed by a mathematical function, a depth map describing the surface from 
one point of view is used. 



Display Processor 

• To achieve a reasonable system performance a special hardware was developed 
■which computes, independently from the CPU, all transformations necessary for 
; displaying a combined image as mentioned. ■ 

iv' ^ 
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Fig. 9: computer system 

An overview of the complete VME-bus based computer system is given in fig. 9. 
Sjnfacc images and CT-data arc stored in the VME-bus RAM. For every pixel of the 
destination image, each one of three "image resampling sequencers" TMC 2301 by TRW 
.^culates one coordinate of the corresponding source voxel of the cutting plane, 
ynchronously another two TMC 2301 calculate the source address of the corresponding 
^lacc image pixel. The virtual depth of the cutting plane for this pixel is calculated by a 
^w^crTMC 2301. A hardware comparator compares the outcome with the surface depth 



taken firom the depth map. This depth map is stored in a local RAM area on the display 
processor to avoid additional VME-bus cycles during image generation. Depending oo 
the result of the depth comparison, either the CT- voxel or the associated surface image 
pixel is read out of the RAM and after compression from 1 6 to 8 bits by an intermal look 
up tablp it is written into the display memory. 

Additional graphical information like the graphical representation of the stylus <r 
the pull-down menus used for user interaction is added by the host CPU. For the CPU, 
the coordinate transformer appears to be completely transparent 



Depth Map Generation 

Since the "image resampling sequencers" build up the destination image column 
by column and row by row, a straightforward ray tracing algorithm can be easily 
implemented. The Coordinate Transformer is programmed to calculate subsequently 
cutting planes through the volume model in parallel to the viewing direction (fig, 10). 




row of depth map 



cutting plane in parallel 
to viewing direction 



search rays 



depth 



Fig. 10: depdi map generation 



The computation of the depth map is supported by the address generator urn 
However, instead of writing the source voxel read-out of the CT-RAM directly to the 
display memory, they are transferred to a hardware grey level comparator. The steps 
within one row of one cutting plane are counted until a previously defined grey level is 



cded The dctcnnincd count is stored at the corresponding location in the depth map. 
^ result each cutting plane supplies one row of the depth map. The depth map 
ted this way may be read out by the host CPU to compute a grey level shaded 

geo 

surface image. 

^Performance of the Display Processor 

The local RAM area on the extension board is sized 512 x 512 x 9 bit, allowing 
" four 256 X 256 bit depth charts to be used simultaneously. The generation of one depth 
Tinap based on a 256xx3 voxel cube requires less than 10 seconds independent from the 
•Viewing direction. The generation of a 256 x 256 image takes about 40ms for a simple 
"sectional view as well as for an image combined of cutting plane and surface image. The 
kjcal grey-level look-up table has a size of 16 x 8 bit Since standard CT-slices have a 
liynamic range of only 12 bit, the upper four bits may be used to change contrast and 
brightness of sectional view and surface image independently. 
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